Towards evaluating crowdsourced image classification on mobile devices to generate geographic information about human settlements

Geographic information crowdsourcing is an increasingly popular approach to derive geographic data about human settlements from remotely sensed imagery. However, crowdsourcing approaches are frequently associated with uncertainty about the quality of the information produced. Although previous studies have found acceptable quality of crowdsourced information in some application domains, there is still lack of research about the quality of information produced with mobile crowdsourcing tools. This paper aims to contribute towards filling this gap by presenting an initial analysis of the contributions of two crowdsourcing projects based on the MapSwipe mobile app, in Madagascar and South Sudan. Our results show, that there is substantial agreement amongst volunteers thus suggesting that mobile crowdsourcing is a viable approach to support the mapping of human settlements. Nevertheless, this study also identifies several factors that may cause disagreement between volunteers (e.g. bad imagery, dependence on individual users) and thus reduce the reliability of the information they produce.